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Abstract 

Since at least the 1980s, motivated, in part, by findings from international 
comparisons of students’ mathematics achievement, some American (U.S.) 
educators and policy makers have initiated educational reform that focuses 
on improving teaching practices and curriculum designs by advocating for 
the adoption of Asian educational models. The implicit assumption behind 
such reform seems to overlook the possible roles of outside of school factors 
in superior academic performance in the Asian countries. The purpose of 
this study was to investigate how outside of school factors were related to 
students ’TIMSS mathematics achievement for the U.S and five top-performing 
Asian nations or regions, and to examine whether the relative role of school- 
associated and outside of school factors was consistent across these five Asian 
countries. The results indicated that students ’achievement was highly related 
to outside of school factors across all six countries. For Singapore and Hong 
Kong, school-associated factors had somewhat more salient roles than the 
outside of school factors. However, for Korea, Taiwan, and Japan, school- 
associated factors played lessor roles in students ’achievement performance. As 
for the US., school-associated factors were related to students ’performance, 
however outside of school factors clearly played more salient roles than the 
school-associated factors. The findings suggest that students ’ mathematics 
achievement covaries substantially with outside of school factors. Hence, 
students ’superior mathematics achievement does not necessarily reflect only 
a superior public school system but also outside of school factors. 
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Introduction 

Since the 1960s, a series of international eomparisons of student 
aehievement have revealed that Ameriean (U.S.) students under-performed in 
mathematies eompared to their eounterparts in other industrialized eountries, 
East Asian eountries inpartieular (Beaton, et. al., 1996; Meknight et al., 1987; 
Robitaille & Garden, 1988; Stevenson et al., 1990). Sinee the 1980s, a great 
number of reports have suggested that Ameriean mathematies edueation is at 
risk and Ameriean students have lagged behind their eounterparts in many East 
Asian eountries. For example, in 1 983, the National Commission on Exeellenee 
in Edueation published ^ at Risk. In 1987, MeKnight et. al. published 

The Underachieving Curriculum: Assessing U.S. School Mathematics from 
an International Perspective. 

Based, in part, on the findings from the international aehievement 
assessments, sueh as the Trends in International Mathematies and Seienee 
Study (TIMSS), there was a strong eall for the reform of the publie sehool 
system. As a result, many U.S. edueators and poliey-makers began to 
undertake edueational reform foeusing on faetors that are eontrollable by 
the sehool systems sueh as eurrieulum design, institutional organization, and 
teaehing standards by advoeating for the adoption of the edueational models 
of the top -performing Asian eountries, (e.g.. National Couneil of Teaehers 
of Mathematies, 1989, 1991, 2000; Interstate New Teaehers Assessment 
and Support Teaehing Consortium, 1992). Sinee the 1980s, the Japanese 
edueational system has been held as a model for the U.S., and in reeent years, 
inereasing attention has been paid to the Singaporean model (DeCoker, 2002; 
Gopinathan, 1997). 

There is a taeit and erueial, nonetheless unverified, assumption behind 
these international eomparison-based reforms. That is, it is taeitly assumed 
that the superior performanee of these Asian eountries is a planned outeome 
of the publie sehool systems, instead of that of the outside of sehool faetors. 
Given that individual, familial, and eultural faetors other than the sehool system 
have been suggested to play essential roles in Asian students’ mathematies 
aehievement, it seems unfounded to attribute Asian students’ superior 
performanee in mathematies solely to the publie sehool system. In other 
words, a diselosure of the integral pieture in depieting students’ mathematies 
aehievement, in and out of sehool, would help edueators and poliey makers 
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reach more effective and realistic decisions. In sum, for the international 
comparison-based school reforms to be meaningful and useful, it is crucial 
to understand the potential role that outside of school factors may play in 
students’ mathematics achievement. 

Study Purpose 

The purpose of this study was to investigate: (1) whether and to what 
extent the outside of school factors, in the presence of school-associated 
factors, were related to students’ mathematics achievement for the U.S. and five 
top-performing Asian nations/regions (i.e., Korea, Taiwan, Japan, Singapore, 
and Hong Kong); (2) whether the relative role of school-associated and outside 
of school factors is consistent across these five top-performing countries. We 
will first briefly review literature on the relation of both in and out of school 
factors related to mathematics achievement and then present the findings of 
the present study addressing these factors with TIMSS 2003 data. 

School-Associated Factors Related to Mathematics Achievement 

In terms of curriculum, a large amount of research has explored the 
differences in mathematics curriculum between the U.S. and some top- 
performing Asian countries. Substantial differences have been found in 
their curricula. Silver (1998) nicely summarizes this review by stating that 
the mathematics curriculum in the U.S. is a mile wide and an inch deep. 
Many researchers indicated that institutional organization has an impact on 
achievement (Greenwald, Hedges, & Laine, 1996; Lee & Bryk, 1989). TIMSS 
researchers have noted that, in the U.S., grade 8 students of different abilities 
are typically grouped into different classrooms and study different materials, 
whereas no such ability grouping is commonly adopted in East Asian countries 
(Lee, 1998; Silver, 1998). School size and classroom size are other factors 
related to students’ access to learning. Lee, Smith, and Croninger (1997) have 
found that larger schools negatively affect students’ mathematics and science 
performance. However, using TIMSS data, other researchers have found that 
larger class size is positively associated with students’ superior performance 
in mathematics for top-performing Asian countries (Silver, 1998). 

Other reform has focused on teaching quality that may have influenced 
the U.S. students’ performance in mathematics. Substantial differences have 
been found in instructional practices in the classroom. American teachers 
have often emphasized skill acquisition that leads to lower-level cognitive 


Spring 2006 / Volume 6, Number 1 


4 

processes, whereas Asian teaehers have always assisted students in obtaining 
deeper understanding, whieh leads students to engage in eonstruetive thinking 
proeesses (Lee, 1998; Silver, 1998). Stigler and Perry (1988) reported that 
Chinese teaehers spent substantially more time on elassroom instruetion than 
did the U.S. teaehers. In addition, some researehers advoeated the whole-elass 
teaehing method implemented by Asian teaehers and suggested that the praetiee 
of redueing elass size in Ameriea was not helpful to improve teaehing-leaming 
quality (Stevenson & Lee, 1995; Stigler & Perry, 1988; Wang & Lin, 2005). 
Furthermore, TIMSS findings have shown that Ameriean teaehers have less 
struetured support for their professional development (Silver, 1998). Chinese 
and Japanese teaehers were organized to take part in professional aetivities, 
sueh as observing and eritiquing eaeh other’s teaehing. However, Ameriean 
teaehers had little ehanee to interaet with their eolleagues and were more likely 
to work in isolation (Lewis, 2000; Paine, 1997; Paine & Ma, 1993). 

Outside of school Factors Related to Mathematics Achievement 

Many researehers have explored whether faetors other than sehool- 
assoeiated faetors have an influenee on students’ performanee. Family 
eharaeteristies (e.g., parents’ edueation and parents’ values) have been suggested 
to have a direet influenee on students’ sehool performanee (Steinberg, Elmen, 
& Mounts, 1989). Parents’ edueation level and their expeetations have been 
demonstrated to be positively assoeiated with students’ performanee (Blair 
& Qian, 1998; Sehreiber, 2000). Also, students’ attitudes and beliefs played 
an important role in their mathematies aehievement (MeLeod, 1992). Many 
researehers have found that positive attitudes are likely to lead to higher 
aehievement, and positive self-eoneepts lead to higher motivation and thus 
to higher learning outeomes (e.g., Ma, 1997; Maslow, 1971; Rogers, 1982). 
However, some researehers have suggested that attitude and self-eoneept are 
ineffeetive predietors of students’ mathematies aehievement based on their 
findings that high performing Asian students have lower self-eoneept and more 
negative attitudes than their U.S. eounterparts. (Howie et al., 2000; Lokan & 
Greenwood, 2000; Shen & Pedulla, 2000; Shen, 2002; Papanastasiou, 2002). 
In addition to family and students’ personal faetors, private tutoring has been 
pervasive in Japan, Korea and Taiwan, and has been shown to have a positive 
effeet on student aehievement (Bender, 1994; Invemizzi, Juel, & Rosemary, 
1997; Lee, 1998). Some researehers have even argued that the pervasive private 
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txitoring is a result of poor public education in these countries (Koskinen & 
Wilson, 1982; Goya, 1994; Johnson & Johnson, 1996). 

Method 

Participants 

Three types of participants were involved in this study: students, the 
students’ teachers, and the students’ school principals. The student participants 
were grade-8 equivalent (approximately 13 years old) students at the time 
of testing in each country. Student sample sizes were: 8,912 (U.S.), 5,309 
(Korea), 4,856 (Japan), 5,379 (Taiwan), 6,018 (Singapore), and 4,972 (Hong 
Kong). The number of students within each school ranged from 20 to 60 for 
Korea, 8 to 43 for Japan, 19 to 55 for Taiwan, 4 to 63 for U.S., 27 to 43 for 
Singapore, and 23 to 188 for Hong Kong. Data from one Japanese and five 
U. S. schools were removed because those schools had less than 10 students. 
The final numbers of principals (which is the same as the number of schools) 
were: 150 (Korea), 146 (Japan), 151 (Taiwan), 165 (Singapore), 126 (Hong 
Kong), and 297 (U.S.). Excluding missing data and teachers who had less then 
10 students, the final teacher sample sizes were: 256 (Korea), 145 (Japan), 
150 (Taiwan), 322 (Singapore), 135 (Hong Kong), and 330 (U.S.). 

Instrument and Variables 

TIMSS 2003 grade-8 mathematics tests were used as our mathematics 
achievement measure. The outcome variable for mathematics achievement 
was the average of five plausible values. Although individual students took 
a test that had some different items from those taken by the other students as 
a result of the matrix sampling, plausible values estimated by item response 
theory scaled students’ performance on a common metric with a mean of 
500 and standard deviation of 100, and hence, made cross-student and cross- 
country comparisons possible. 

Along with the TIMSS achievement test administration, comprehensive 
background questionnaires were administered to the students, students’ 
teachers, and the school principals. The student’s questionnaire asked about 
their home backgrounds, their attitudes toward learning mathematics, and 
their experiences in learning mathematics. The teacher’s questionnaire asked 
teachers about their teaching preparation, teaching activities and approaches, 
attitudes toward teaching, as well as the curriculum covered in the classroom. 
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The principal’s questionnaire asked school principals to provide information 
about curricular and instructional arrangements, school resources, and school 
climate. These questionnaires were used in the present study to identify 
potential school-associated variables and outside of school variables that are 
related to the students’ achievement. As a result of our literature review and 
empirical correlation analyses on TIMSS data, a set of variables was selected 
to study their relationships with mathematics achievement. These variables 
included items in the questionnaires originally answered by the participants 
as well as “indices” and “derived variables” reproduced by TIMSS (See 
Supplement 3, TIMSS 2003 User-guide). The selected variables were classified 
into school-associated variables and outside of school variables. 

It is crucial to note that our categorization of school-associated and 
outside of school factors was based on the TIMSS data collection design. 
That is, school-associated variables refer to variables collected from the 
teacher’s and principal’s questionnaires, and outside of school variables refer 
to variables collected from the student’s questionnaire. The outside of school 
factors are comprised of individual, familial, and cultural factors. The school- 
associated factors mcludQd school context factors, such as school climate, and 
school practice factors, such as curriculum coverage (Raudenbush & Willms, 
1995). 

As a result of our classification, eight outside of school variables 
reflecting students’ individual, familial, and cultural differences were included 
in our study. The school-associated variables included: seven principal 
variables from the principals’ report on schools’ contexts and practices, and 
twenty-four teacher variables from the teachers’ report on teaching practices 
in the classroom and professional development. All of the variables, including 
school-associated and outside of school, are listed and described in Table 1 . 

Data Analyses 

Two classes of statistical methods were adopted to answer our 
research questions. Method- 1 utilized hierarchical linear modeling (HLM) 
(Raudenbush & Bryk, 2002). Method-2 utilized a variable ordering 
technique, the relative Pratt index (Pratt, 1987; Thomas, Hughes, & Zumbo, 
1998; Thomas & Zumbo, 1996), used in multiple regression analyses. The 
TIMSS data were structured as follows: students were nested within teachers, 
and teachers were nested in schools and hence, students with the same 
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Table 1 


Description of Variables Used in HLM Analyses 


Variables 

Code or 
range 

Description 

Student-level Variables 

Outcome variable 


The average of five plausible values of mathematies. 

Parent educational level 
(PARENTEDU) 

1-5 

A variable derived from students’ responses to the highest 
education level of both mother and father (1 = no more than 
primary sehooling, 2 = finish lower seeondary sehooling, 

3 = finish upper secondary schooling, 4 = finish post- 
secondary vocational/technical education, 5 = finish univ. 
or higher). 

Educational aspirations 
(ASP) 

1-4 

A variable derived from students’ responses to how far in 
sehool they expeet to go relative to parents’ edueational 
level (1 = do not know regardless of parents’ education, 2 = 
not finish univ. regardless of parents’ education, 3 = finish 
univ. but neither parent went to univ., 4 = finish univ. and 
either parent went to univ.). 

Students’ self-confidence 
in learning mathematics 
(SLCONF) 

1-3 

An index shows students’ self-confidence in learning 
mathematics and is derived from student’s responses to 
four questions regarding mathematies: (a) I usually do well 
in math, (b) math is more difficult for me than for many of 
my classmates, (c) math is not one of my strengths, and (d) 
I learn things quickly in math. (1 = low, 2 = medium, 

3 = high). 

Students’ valuing 
mathematics (VALUE) 

1-3 

An index shows students valuing the importanee of math, 
which is derived from seven questions. (1 = low, 

2 = Medium, 3 = high). 

Time students spend 
doing math homework 
(TIMHW) 

1-3 

A variable is derived from 2 questions: (a) how often your 
teaeher gives you homework in math, and (b) when your 
teacher gives you math homework and how many minutes 
are you usually given. (1 = low, 2 = medium, 3 = high). 

Extra lessons or tutoring 
(TUTORING) 

1-4 

An original variable from student questionnaire to indicate 
the frequency of extra lessons or tutoring in mathematics 
(1 = never, 2 = sometimes, 3 = once or twice a day, 

4 = every day). 

Availability of eomputer 
(AVABLCOMP) 

1-5 

A derived variable from two questions: (a) Do you ever use 
a eomputer, and (b) Where do you use a computer (1 = Do 
not use at all, 2 = only at places other than home and 
school, 3 = at school but not at home, 4 = at home but not 
at sehool, 5 = both at home and sehool). 

Number of books at home 
(NRBOOK) 

1-5 

(1 = 0 - 10, 2 = 1 1 - 25, 3 = 26 - 100, 4= 101 - 200, 
5 = more than 200). 
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School-level Variables 

Principal variables 

School size (SCHSIZE) The enrollment of total students in the sehool. 


Good sehool and class 
attendance (GOODSCH) 

1-3 

A eomposite index based on (a) how often the following 
problem behaviors occur in the sehool, such as arriving 
late, absenteeism, skipping class, classroom disturbance, 
cheating, profanity, and vandalism and (b) if the behavior 
oecurs, how severe a problem does it present (1 = low, 

2 = medium, 3 = high). 

Sehool elimate 
(SCHCLIMATE) 

1-3 

A composite index based on prineiples’ response to eight 
questions regarding sehool elimate, including teachers’ job 
satisfaction, understanding of the school’s curricular goals, 
degree of success in implementing the school’s curriculum, 
and expectations for student achievement, and involving 
parental support and students’ regard for school property 
(1 = low, 2 = medium, 3 = high). 

Low social economic 
status (LOWSES) 

1-4 

Percentage of students in the school who come from 
economieally disadvantage home (1=0- 10%, 2=11- 
25%, 3 = 26 - 50%, 4 = more than 50%). 

High social economic 
status (HIGHSES) 

1-4 

Pereentage of students in the sehool who eome from 
economieally affluent home (1 = 0- 10%, 2 = 11- 25%, 
3 = 26 - 50%, 4 = more than 50%). 

Grouping instruction 
(GROUPCURRICULUM) 

1-3 

Organization of currieulum for grade 8 students with 
different levels of ability (1 = study same eurriculum, 

2 = same currieulum, but at different levels of difficulty, 

3 = different math eurricula aeeording to their ability). 

Grouping students 

0 = No, 

Grouping students within their math elasses by their levels 

(GROUPSTUD) 
Teacher variables 

1 = Yes 

of ability. 

Pereeption of no or few 
limitations on instruetion 
due to students 
(PERCPLIMIT) 

1-3 

An index of teaehers' reports on teaching math classes with 
few or no limitations on instruetion due to student factors 
(i.e., students with different aeademie abilities, from a wide 
range of backgrounds, with special needs, uninterested 
student, ete.) 1 = a lot of limitation, 2 = some limitation, 3 
= no limitation or little). 

Emphasis on homework 
(EMHW) 

1-3 

This index is eomputed from two items: (a) do you assign 
math homework to the TIMSS class, (b) how many minutes 
do you usually assign (3 = high, 2 = medium, 1 = low). 

Class size (CLASIZE) 

1 -4 

The number of student in TIMSS elasses (1 = 1- 24, 
2 = 25 - 32, 3 = 33 - 40, 4 = above 41). 

Covering overall math 
topics (MATHTOPICS) 

1-3 

This index deseribes when TIMSS class students have been 
taught over topies of number, geometry, algebra, data, and 


measurement (percentage of overall math topics covered) 
(1 = not yet taught or just introdueed, 2 = most taught this 
year, 3 = mostly taught before this year). 
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Teaching time 
(TCHTIME) 

Interaction with 1-4 

colleagues 

(INTERACTION) 

Professional development 0 = No, 

(PROEDEVEEOP) 1 = Yes 

Content related activities 1-4 

(CONTENTACT) 


This item indicates how many minutes per week the 
teachers teach math to the TIMSS class. 

Eour individual items describe how often teachers have 
four types of interactions with other teachers (1 = never, 2 
= 2 or 3 times/month, 3 = 1-3 times/week, 4 = daily or 
almost daily). 

Six individual items indicate whether teachers have 
participated in various types of professional development in 
the past two years. 

Nine individual items indicate the frequency with which 
the teacher asks students to do various content-related 
activities in math (1 = never, 2 = some lessons, 3 = about 
half the lessons, 4 = every lesson). 


teachers and the same school were more likely to share the same context 
than students from different teachers or schools. In this sense, the clustered 
data may violate the assumption of independence of all observations found 
in ordinary least-squares regression. Therefore, HLM analyses were utilized 
to deal with the clustered data. The second method, the relative Pratt index, 
allowed us to investigate school and outside of school effects using a single 
model including all factors. However, both statistical procedures had their 
own advantages as well as limitations, which are described below. 

Method-1. For TIMSS data, which was clustered by data collection 
design, HLM analyses had the advantage of increasing the precision of the 
coefficient estimates of school level variables and the power of hypothesis tests 
of the coefficients by controlling for the variation in student level variables. 
This advantage is especially the case if the student level factors were strongly 
related to the outcome variable (Lee & Bryk, 1989; Bachmann & Homung, 
2003). The amount of dependence of observations can be expressed as the 
intraclass correlation, which can be estimated in the unconditional multilevel 
model — that is, the model only with intercept and residual error term. The 
absence of intraclass correlation indicates that there is no design effect caused 
by clustered data. However, Kreft and de Leeuw (1998) have indicated that 
even a small intraclass correlation of .01 can substantially inflate the Type I 
error. Hence, the intraclass correlation was reported in the section of results 
to indicate the necessity of using HLM technique for the present study. 

For the TIMSS data collection design, students (level-1) were nested 
within teachers (level-2) and teachers were nested in the schools (level-3), 
which normally involves a 3 -level HLM to appropriately account for the 
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variation of the nested data. However, a 3 -level model was not adopted for 
this study beeause of the insufheient teaeher sample size at level-2 (i.e., only 
one or two teaehers were nested in eaeh sehool). An intuitive way to remedy 
this problem was to aggregate the seores for the teaeher variables to the 
sehool level by taking the average of the teaehers. However, eonsidering 
that seores based on one teaeher or the average of the two teaehers were not 
representative of the sehool, this approaeh was not adopted. Instead, two 
2-level analyses were modeled: (1) Student-teacher Model, whieh involved 
the estimation of variation between students’ mathematies aehievement 
nested within teaehers, and (2) Student-principal Model, whieh involved the 
estimation of variation between students’ mathematies aehievement nested 
in the sehools. The strueture of these 2-level models is presented in Figure 1 . 

Figure 1 

The Structure of Two 2-level Hierarchical Linear Models 


Student-principal Model 


Level 1 
Level 2 


T 

Model A 


T 

Model B 


T 

Model C 


None Outside-of-school Variables Outside-of-school Variables 

None None School-associated Variables 


Student-teacher Model 



T 

T 

T 


Model A 

Model B 

Model C 

Level 1 

None 

Outside-of-school Variables 

Outside-of-school Variables 

Level 2 

None 

None 

School-associated Variables 
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The linearity assumption on the level- 1 outeome variable was examined 
and shown to be appropriate for our data. Following the suggestions by 
Raudenbush and Bryk (2002), all predietors at level- 1 were eentered about 
their means for ease of interpretation. For parameter estimation, “restrieted 
maximum likelihood” (RML) method was used. The various models will be 
introdueed later in the results seetion. 

Method-2. To remedy the inability to ineorporate both teaeher and 
prineipal variables in one HLM model in method- 1, a multiple regression 
analysis was eondueted and then the Relative Pratt indiees (Thomas, Hughes, 
& Zumbo, 1 998) were ealeulated to order the relative importanee of outside of 
sehool and sehool-assoeiated variables. Using eriteria deseribed by Thomas 
et al. as well as Thomas and Zumbo (1996), variables that aeeounted for R- 
squared > 0.02 were regarded as important variables; this is a eonservative 
implementation of Thomas et al.’s eriteria for importanee, \!2p where p 
is the number of predietors in the regression model. The limitation of the 
multiple regression approaeh was that the test of statistieal signifieanee for 
the regression parameters (and the overall model) would be inflated due to 
the design effeet, the intraelass eorrelation. It should be noted, however, that 
the Pratt indiees are deseriptive and do not involve statistieal signifieanee, 
so the design effeet problem is less worrisome for investigating the relative 
importanee of variables. 

Results 

The deseriptive statisties of the outeome variables (TIMSS 2003 
mathematies seore) are reported in Table 2. Reporting of the Asian eountries 
was ordered by TIMSS international rankings (from top one to top five) aeross 
46 partieipating eountries. The U.S. students’ seore was slightly above the 
international average and was ranked as 14^^ plaee. 

Statistical Method-1: Hierarchical Linear Modeling 

In the next paragraphs, three HLM models are presented to investigate 
the relationships of in and out of sehool faetors to students’ mathematies 
aehievement - referred to as Model- A, Model-B, and Model-C throughout. 
Beeause effeets of sehool-assoeiated faetors were investigated separately 
by teaeher and prineipal variables, eaeh of Models A-C sehool effeets were 
examined by two separate 2-level models: model for students nested within 
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Table 2 


Descriptive Statistics ofTIMSS Grade-8 Students’ Mathematics 
Achievement 


Nations 

N 

Mean 

SD 

Min. 

Max. 

SGP 

6018 

602 

77 

349 

804 

KOR 

5379 

589 

81 

281 

846 

HK 

4972 

588 

68 

291 

752 

TWN 

5379 

587 

97 

265 

831 

JPN 

4856 

569 

77 

274 

833 

US 

8912 

504 

78 

263 

738 


teachers (i.e., student-teacher model) as well as model for students nested in 
schools (i.e., student-principal model). 

Unconditional Model (Model-A) 

The 2-level unconditional model served to partition the total variance 
in mathematics achievement into within school (i.e., the effect of outside of 
school variables) and between school (i.e., the effect of school-associated 
variables) components. The unconditional model included a random 
intercept and a level- 1 residual rj. The level- 1 model was specified as: 

where i denotes individual students and j denotes schools, and the random 
intercept of level- 1 was the outcome of the level-2 model, 

Poj~ Too ^oj ’ 

where Too was the grand mean of mathematics scores for the school-level 

distribution and u^j was the variation between schools. This model is equivalent 
to a one-way ANOVA with random effects. 

The results of the unconditional model were presented in Table 3. In 
terms of the student-principal model, the estimated average mathematics 
scores across schools were 588.2, 567.9, 584.7, 502.2, 601.7, and 584.1 for 
Korea, Japan, Taiwan, U.S., Singapore, and Hong Kong, respectively. Both 
student-level and school-level residual variances were statistically significant, 
suggesting that there was a need to add predictors to account for the left-over 
variances at both levels. The comparisons of the intraclass correlations across 
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six countries indicated that substantial proportions of variance in average 
mathematics achievement could be accounted for by school-level variables 
for U.S., Singapore, and Hong Kong (i.e., 0.43, 0.41, and 0.62). However, for 
Korea, Japan, and Taiwan, less proportions of variance in average mathematics 
achievement can be accounted for by school-level variables indicated by the 
relatively lower intraclass correlations, (i.e., 0. 10, 0. 15, and 0.30). The results 
of the student-teacher model showed similar patterns as the student-principal 
model except for slight differences in the parameter estimates. 


Table 3 

Multilevel Estimates for Unconditional Model (Model-A) 

Parameter Coefficient 




KOR 

JPN 

TWN 

u.s. 

SGP 

HK 

Student-principal Model 

Fix effect 
Intercept 

/ 00 

588.2*** 

567.9*** 

584.7*** 

502.2*** 

601.7*** 

584.1*** 

Random effect 
Level- 1 variance 


5976*** 

494J*** 

6569*** 

3455*** 

3478*** 

1906*** 

Leve-2 variance 

^uo 

627*** 

875*** 

2872*** 

2568*** 

2420*** 


Intraclass correlation 


0.1 

0.15 

0.3 

0.43 

0.41 

0.62 

Goodness-of-fit 

Deviance 

62266 

55358 

62961 

98669 

66682 

52178 

Student-teacher Model 








Fix effect 








Intercept 

Too 

590.3*** 

567.9*** 

584.2*** 

500.7*** 

599.5*** 

580.0*** 

Random effect 
Level- 1 variance 

2 

5945*** 

494J*** 

6542*** 

2535*** 

1322*** 

1864*** 

Leve-2 variance 

*^//0 

626*** 

875*** 

2930*** 

3856*** 

4716*** 

3483*** 

Intraclass correlation 


0.1 

0.15 

0.31 

0.6 

0.78 

0.65 

Goodness-of-fit 

Deviance 

62322 

55358 

62946 

96453 

61633 

52137 


Note: *** denotes OC significance level at .001, ** at .01, * at .05. 


Outside-school-effects-only Model (Model-B) 

In the second stage, we began to test the relationship between the eight 
outside of school factors and students’ mathematics achievement. The random 
intercept and slope model, which is also called a random-coefficient regression 
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model by Raudenbush and Bryk (2002), was adopted beeause the slopes of 
level- 1 variables were hypothesized to vary aeross sehools. Thus, the eentral 
eoneem in Model B was to test if the slopes of level- 1 faetors were random 
aeross sehools for eaeh of the six eountries. If the slopes of level- 1 variables 
were not varying aeross sehools, then they should be fixed. For Model-B, 
level- 1 and level-2 models were speeified as: 

Yy = /?o; + AyPARENTEDU+ ASP + (i^. VALUE + (i,. SLCONF 

+ TMHW + TUTORING + . AVABLCOMP + ^^g^NRBOOK + ^. , 

Poj~ y 00 ^oj ’ 

/^pj= rpo + i^pj (;?=h2, 8). 

The results of Model-B analyses showed that for some eountries, 
allowing the level- 1 slopes variant aeross sehools were mis-speeified and 
failed to aehieve eonvergenee. Therefore, there was a need to fix some non- 
signifieant varianees and all eovarianees of level- 1 faetors. The final results 
of partly-fixed-partly-random level- 1 slope model are presented in Table 4. 

In terms of the student-prineipal model, the slope of all eight level- 
1 variables were fixed for Japan, and nearly all of them were fixed for 
Korea, Taiwan, and Hong Kong. This indieated that there was no eross- 
sehool variation in the relationships between all outside of sehool faetors 
and mathematies aehievement for Japan. Cross-sehool variation in the 
aehievement and predietor relationship only existed in parents’ edueation 
(PARENTEDU) and students’ valuing the importanee of mathematies 
(VALUE) for Korea, only in students’ aspiration relative to parents’ 
edueation (ASP) and tutoring time in mathematies (TUTORING) for Taiwan, 
only in students’ self-eonfidenee in learning mathematies (SLCONF) and 
time spending on mathematies homework (TMHW) for Hong Kong. On 
the eontrary, the slopes of level- 1 variables were mostly random for U.S. 
and Singapore. The varying slopes of student-level variables in U.S. and 
Singapore indieated that outside of sehool variables varied aeross sehools. 

In terms of the student-teaeher model, the relationship between 
mathematies and the eight outside of sehool faetors were eonsistent with 
the findings of the student-prineipal model for Japan. For Korea and Taiwan, 
only the relationships between mathematies aehievement and ASP varied 
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Table 4 


Multilevel Estimates for Model-B 



Parameter 



Coefficient Estimates 



Student-principal Model 

Fix effect 


KOR 

JPN 

TWN 

u.s. 

SGP 

HK 

Intercept 

y QQ 

587.8*** 

567.3*** 

592 9*** 

469 4*** 

602.9*** 

575.2*** 

PARENTEDU 

rw 

0.8 


2.2* 

2 g*** 

0.8 

-1.0 

ASP 

y 20 

12.0*** 



5.8* 

6.3*** 

0.3 

VALUE 

y 20 


5 3*** 

6.8*** 

_2 2*** 

g 5*** 

6.0*** 

SLCONF 

y AO 

40.3*** 

35.0*** 

42 4*=i=!|= 

22 4*H«H« 

22.2*** 

24 4*** 

TMHW 

Y 50 

-2.9* 

-10 4*** 

4.2** 

9 2*** 

Y2 5*** 

0.9 

TUTORING 

yoo 

_4 2*** 

3.3** 

-5.8*** 

18.3*** 

3.1* 

2 2*** 

AVABLCOMP 

y 70 

12.2*** 


9 9^4** 

5 9*** 

10.0*** 

4 

NRBOOK 

y$o 

12.0*** 

2 

^ y 2*** 

g 4H«H«H« 

^ 4Hshshs 

2 2*** 

Random effect 

Variance 

Component 







Level- 1 variance 


3164*** 

3044*** 

3686*** 

2185*** 

2222*** 

1320*** 

Leve-2 variance 



622*** 

2 292*** 

1356*** 

1318*** 

2562*** 

PARENTEDU 


38.3* 

- 

- 

- 

23 2*** 

- 

ASP 


- 

- 

101.5*** 

42.8*** 

140*** 

- 

VALUE 

<yl2 

73.8* 

- 

- 

52.6* 

74.1** 

- 

SLCONF 


. 

- 

- 

61.8*** 

251.3** 

35.3* 

TMHW 


_ 

_ 

_ 

262.5*** 

144*** 

4QH«H«* 

TUTORING 


_ 

_ 

36.9* 

27.1** 

_ 

_ 

AVABLCOMP 


. 

- 

- 

- 

- 

- 

NRBOOK 


_ 

_ 

_ 

_ 

29^*** 

_ 

Goodness-of-fit 

Deviance 

54361 

37259 

55560 

73186 

53006 

43144 

Student-teacher Model 


KOR 

JPN 

TWN 

U.S, 

SGP 

HK 

Fix effect 








Intercept 

y 00 

588.6*** 

567.3*** 

591 4*** 

472.6*** 

592 2*** 

^22 2*** 

PARENTEDU 

ym 

0.8 


2.1* 

2 2*** 

-0.3 

-1.0 

ASP 

y 20 

^2 5*** 

^2 4*** 

j^2 2*** 

3.6*** 

-0.1 

0.6 

VALUE 

^30 

10.3*** 

5 3*** 

6.8*** 

-1.7 

g 2*** 

5.8*** 

SLCONF 

^40 

40.2*** 

35*** 

43.6*** 

24 4*** 

1§ 4*=i==i= 

24 3*H«s!« 

TMHW 

yso 

-2.3 

-10 4*** 

4 4*Hs 

2.1 

1.9* 

1.2 

TUTORING 

y 60 

_4 

3.3** 

-5.6*** 

2*** 

3 4H«s!«H« 

2 

AVABLCOMP 

y 70 

11.5*** 

2 2*** 

9 9^4** 

4 2*** 

4 

4 3*** 
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NRBOOK 

7 m 

12.2*** 


Y 3*=I=!|= 


0.9* 


Random effect 
Level- 1 variance 


3172*** 

3044*** 

3718*** 

1686*** 


1274*** 

Leve-2 variance 


264*** 

622*** 

1241*** 

2328*** 

4064*** 

2842*** 

PARENTEDU 


- 

- 

- 

- 

- 

- 

ASP 


54.3* 

_ 

101.6*** 

19.0* 

_ 

40.2* 

VALUE 


_ 

_ 

_ 

39.3* 

39.5* 

_ 

SLCONF 


_ 

_ 

_ 

32.3** 

_ 

39.0** 

TMHW 


_ 

_ 

_ 

122.7 

32.2* 

46.3* 

TUTORING 


. 

. 

. 

26.3** 

66.9*** 

. 

AVABLCOMP 


_ 

_ 

_ 

32.0* 

_ 

_ 

NRBOOK 


_ 

_ 

_ 

14.5** 

_ 

_ 

Goodness-of-fit 

Deviance 

54404 

37259 

55564 

71941 

49278 

43127 


Note: *** denotes a, significance level at .001, ** at .01, * at .05. 


across classrooms. However, the relationships between mathematies and all 
the outside of sehool faetors varied aeross elassrooms for U.S., Singapore, 
and Hong Kong. As a result of the above proeedures for fixing the non-random 
effeets, Model-B eonverged for all six eountries, both student-prineipal and 
student-teaeher models. 

The results of Model-B showed that most of the outside of sehool 
variables were statistieally signifieantly and positively related to students’ 
performanee exeept that a few variables had a negative relationship with the 
outeome variable for some eountries, sueh as time spending on mathematies 
homework in the student-prineipal model (-2.9 for Korea and -10.4 for Japan). 
Students’ self-eonfidenee in mathematies ability (SLCONF), tutoring time 
in mathematies (TUTORING), availability of eomputer (AVABLCOMP), 
and the number of books at home (NRBOOK) were found to be signifieant 
positive indieators for both student-prineipal and student-teaeher models. A 
few outside of sehool variables were not statistieally signifieant for some 
eountries. For example, parent edueational level was not a signifieant 
predietor for Korea (0.8), Singapore (-0.3), and Hong Kong (-1.0) in the 
student-prineipal model. Although some outside of sehool variables were 
shown non-signifieant for some eountries, they were kept in the model in 
the next stage for the purpose of eross-eountry eomparisons. Therefore, all 
the outside of sehool variables were used in the next model - “sehool-efifeet 
model”. 
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School-effects Model (Model-C) 

To investigate whether sehool-associated faetors were still significantly 
related to mathematics achievement after controlling for outside of school 
factors, the school-effects models were examined by adding school-level 
variables to explain the variation in the intercepts at level-2. Level- 1 and 
level-2 models were specified as: 

Y,= Ay + AyPARENTEDU + Ay ASP + A, VALUE + Ay SECONF 
+ Ay TMHW + Ay TUTORING + AyAVABECOMP + AyNRBOOK+y,, 

Ay= 7m + Yak +/'oy , (k = 1 , 2, . . . 7 for Studcnt-principal Model; k= 1,2, . . 

24 for Student-teacher Model), Ay= Ypo + ^pj or Ay= (/> =L 2, 8 & some 

of Upj were fixed zero, see Table 4) where denotes variables. 

The results of Model-C with parameter estimates are provided in Tables 
1 A (student-principal model) and 2 A (student-teacher model) in the Appendix. 
The results of school and outside of school factor effects (Model-C) were 
summarized in Table 5 (student-principal model) and Table 6 (student-teacher 
model). In the summary tables, we only presented the results of hypothesis 
tests and the direction of the relationships, because our purpose was to find 
the significant factors, school or outside of school factors, which are common 
and/or inconsistent across the six countries. 

Effects of outside of school variables. The results indicated that outside 
of school factors were substantially related to students ’ achievement for Korea, 
Japan, Taiwan, and U.S., but were relatively less related for Singapore and 
Hong Kong. In addition, the results showed that after including school-level 
variables, a few of student-level variables were no longer associated with 
mathematics achievement, such as parental education levels (PARENTEDU) 
for Taiwan in both student-principal and student-teacher models, and time 
spent doing mathematics homework (TMHW) for Singapore in the student- 
teacher model. 

Despite that, most parameter estimates for the student-level variables 
were still statistically significant for Korea, Japan, Taiwan, and U.S. when 
compared to Model-B results. In both student-principal and student-teacher 
models, students’ self-confidence in learning mathematics (SLCONF) was 
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positively related to mathematies aehievement aeross all six nations. Student’s 
valuing the importanee of mathematies (VALUE) was a positive indieator of 
mathematies aehievement for all five Asian nations, but was negatively related 
to mathematies aehievement for the U.S. in the student-prineipal model and 
was not a signifieant indieator for U.S. in the student-teaeher model. Tutoring 
time (TUTORING) was positively related to aehievement for Japan, U.S., 
Singapore, and Hong Kong, but negatively related to aehievement for Korea 
and Taiwan. In addition, the availability of eomputers at sehool and home 
(AVABLCOMP) was a positive indieator aeross six nations in the student- 
prineipal model. 

Effects of school-level variables (Student-principal Model). The results 
showed that none of the sehool-assoeiated variables were related to students’ 
aehievement for Japan, however, various number of sehool-assoeiated 
variables were related to mathematies aehievement for other eountries. 
The findings displayed somewhat different patterns from that found in the 
literature. The pereentage of students from low soeial elass (LOWSES) was 
related to students’ aehievement for Korea, Taiwan, and U.S., but was not 
related to aehievement for Japan, Singapore, and Hong Kong. Good sehool 
and elass attendanee (GOODSCH) was positively related to mathematies 
aehievement only for U.S. and Hong Kong, but unrelated to aehievement for 
other eountries. Grouping students in their mathematies elass by their ability 
(GROUPSTUD) was not related to mathematies aehievement for all nations. 
Organization of eurrieulum for students with different levels of ability 
(GROUPCURRICUEU) was not related to mathematies aehievement for all 
eountries exeept Hong Kong in whieh it was negative related to aehievement. 
Note that there was no value for Singapore on GROUPCURRICUEUM, 
beeause all Singapore teaehers indieated that they used different eurrieulum for 
their students. Sehool size (SCHSIZE) was positively related to aehievement 
only for Korea, Singapore and Hong Kong. 

Effects of school-level variables (Student-teacher Model). In terms of 
teaeher variables. Table 6 shows that most of them were not assoeiated with 
mathematies aehievement aeross six nations, but eompared to Korea, Japan, 
and Taiwan, more teaeher variables were related to aehievement for U.S., 
Singapore, and Hong Kong. Teaeher ’s emphasis on mathematies homework 
(EMHW) and eoverage of mathematies topies (MATHTOPICS) were positive 
related to aehievement for U.S., Singapore, and Hong Kong, but unrelated 
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to achievement for Korea, Japan, and Taiwan. Class size (CLASIZE) was 
positively related to students’ aehievement for Korea and Hong Kong, but 
not for other eountries. Teaeher’s pereeption of teaehing mathematies elasses 
with few or no limitations on instruetion due to student faetors (PERCPLIMT) 
was positively related to mathematies aehievement in all eountries exeept 
Japan. In addition, most “teaeher’s interaetion with their eolleagues” 
(INTERACTION) and “involvement in professional development aetivities” 
(PROFDEVELOP) variables were unrelated to mathematies aehievement 
aeross the six nations. 


Table 5 


The Summery of Parameter Estimates of Model-C across Six Nations or 
Regions (Student-principal Model) 

School-effects Models 


Parameter Estimates 

KOR 

JPN 

TWN 

u.s. 

SGP 

HK 

PARENTEDU 

NS 

(+) 

NS 

(+) 

NS 

(-) 

ASP 

(+) 

(+) 

(+) 

(+) 

(+) 

NS 

VALUE 

(+) 

(+) 

(+) 

(-) 

(+) 

(+) 

SLCONF 

(+) 

(+) 

(+) 

(+) 

(+) 

(+) 

TMHW 

(-) 

(-) 

(+) 

(+) 

(+) 

NS 

TUTORING 

(-) 

(+) 

(-) 

(+) 

(+) 

(+) 

AVABLCOMP 

(+) 

(+) 

(+) 

(+) 

(+) 

(+) 

NRBOOK 

(+) 

(+) 

(+) 

(+) 

(+) 

NS 

LOWSES 

(-) 

NS 

(-) 

(-) 

NS 

NS 

HIGHSES 

NS 

NS 

(+) 

(+) 

NS 

NS 

GROUPCURRICULUM 

NS 

NS 

NS 

NS 


(-) 

GROUPSTUD 

NS 

NS 

NS 

NS 

NS 

NS 

SCHCLIMATE 

(+) 

NS 

NS 

NS 

(+) 

NS 

GOODSCH 

NS 

NS 

NS 

(+) 

NS 

(+) 

SCHSIZE 

(+) 

NS 

NS 

NS 

(+) 

(+) 


Statistical Method-2: Variable Ordering 

For the purpose of examining sehool-assoeiated and outside of sehool 
variables in a single model, and henee allowing us to order the sehool and 
outside of sehool variables in terms of their relative importanee, we eondueted 
multiple regression analyses and used the Relative Pratt Index (Thomas, 
Hughes, & Zumbo, 1 998; Thomas & Zumbo, 1 996) to partition the proportion 
of varianee aeeounted for by the eight outside of sehool variables and the 3 1 
sehool-level (teaeher and prineipal) variables. 
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Table 6 

The Summery of Parameter Estimates of Model-C across Six Nations or 


Regions (Student-teacher Model) 


School-effects Models 

Parameter Estimates 

KOR 

JPN 

TWN 

u.s. 

SGP 

HK 

PARENTEDU 

NS 

(+) 

NS 

(+) 

NS 

NS 

ASP 

(+) 

(+) 

(+) 

(+) 

NS 

NS 

VALUE 

(+) 

(+) 

(+) 

NS 

(+) 

(+) 

SLCONE 

(+) 

(+) 

(+) 

(+) 

(+) 

(+) 

TMHW 

NS 

(-) 

(+) 

NS 

NS 

NS 

TUTORING 

(-) 

(+) 

(-) 

(+) 

(+) 

(+) 

AVABLCOMP 

(+) 

(+) 

(+) 

(+) 

(+) 

NS 

NRBOOK 

(+) 

(+) 

(+) 

(+) 

NS 

(+) 

PERCPLIMIT 

(+) 

NS 

(+) 

(+) 

(+) 

(+) 

EMHW 

NS 

NS 

NS 

(+) 

(+) 

(+) 

CLASIZE 

(+) 

NS 

NS 

NS 

NS 

(+) 

MATHTOPICS 

NS 

NS 

NS 

(+) 

(+) 

(+) 

TCHTIME 

NS 

(+) 

NS 

NS 

(-) 

NS 

INTERACTION (discuss 
concept) 

INTERACTION (prepare 
instruction material) 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

INTERACTION (observe 
others’ teaching) 

NS 

NS 

NS 

(-) 

NS 

NS 

INTERACTION (observation of 
teaching by others) 

NS 

NS 

NS 

NS 

NS 

NS 

PROEDEVELOP (content) 

NS 

NS 

NS 

NS 

NS 

NS 

PROEDEVELOP (pedagogy) 

NS 

NS 

NS 

NS 

(-) 

NS 

PROEDEVELOP (curriculum) 

NS 

NS 

NS 

NS 

(+) 

NS 

PROEDEVELOP (technology) 

NS 

NS 

(-) 

NS 

NS 

NS 

PROEDEVELOP (improve 
students’ problem solving) 

NS 

NS 

NS 

NS 

NS 

NS 

PROEDEVELOP (assessment) 

NS 

NS 

NS 

NS 

NS 

NS 

CONTENTACT (adding) 

(+) 

NS 

NS 

NS 

NS 

NS 

CONTENTACT (fractions) 

NS 

NS 

NS 

NS 

NS 

NS 

CONTENTACT (no obvious 
solution) 

NS 

NS 

NS 

NS 

NS 

NS 

CONTENTACT (interpret data 
in tables, etc.) 

NS 

NS 

NS 

NS 

(-) 

NS 

CONTENTACT (write 
equations and functions to 
represent relations) 

NS 

NS 

(+) 

NS 

(+) 

NS 

CONTENTACT (work in small 
groups) 

NS 

NS 

NS 

NS 

NS 

NS 

CONTENTACT (relate to daily 
life) 

CONTENTACT (explain their 
answers) 

NS 

NS 

NS 

(-) 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

NS 

CONTENTACT (decide own 
procedures for solving complex 
problems) 

NS 

NS 

NS 

(+) 

(+) 

NS 
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The i?-squared values for eaeh eountry are reported at the bottom of 
Table 7. The i?-squared values ranged from .40 (KOR) to .61 (HK). Given 
that variables refleeting the students’ eognitive abilities were not ineluded in 
the model (TIMSS did not measure these variables), this range ofR-squared 
values were satisfaetory for the purpose of this study. The Relative Pratt 
Indiees that partitioned the explained varianee into sehool and outside of sehool 
eomponents are presented in Figure 2 and Table 8. The results indieated that 
outside of sehool faetors explained a large proportion of varianee, ranging from 
36% to 94% aeross six eountries, but the explained varianee by in and out of 
sehool faetors showed differing patterns aeross five Asian eountries. Figure 2 
shows that for KOR, TWN, and JPN, outside of sehool faetors aeeounted for 
a substantial proportion of the explained variation in students’ aehievement, 
94% (KOR), 89% (TWN), and 87% (JPN). As for U.S., sehool-assoeiated 
faetors aeeounted for slightly more of the explained variation (29%) eompared 
to KOR, TWN and JPN, despite the faet that outside of sehool faetors still 
aeeounted for 71% of the explained varianee. For SGP and HK, sehool- 
assoeiated faetors aeeounted for quite a large proportion of explained variation, 
59% (SGP) and 64% (HK). The results eehoed those found in HLM Method- 1 , 
in whieh Singapore and Hong Kong exhibited quite different patterns in sehool 
and outside of sehool effeets from that of the other three Asian eountries. The 
eross-eountry pattern of sehool vs. outside of sehool faetors shown in Figure 
2 was eonsistent with those revealed by the intraelass eorrelations in Method- 
1 (Table 3). Figure 2 showed that sehool-assoeiated variables aeeounted for 
a small proportion of the explained varianee, .06 (KOR), .13 (JPN), and .11 
(TWN). Eehoing Figure 2, the intraelass eorrelations were . 1 0, . 1 5, and .30 for 
KOR, JPN, and TWN, indieating that sehool-assoeiated variables aeeounted 
for a small proportion of varianee in average mathematies seores; however, 
sehool-assoeiated variables aeeounted for a large proportion for US, SGP and 
HK. 

In order to determine whieh sehool or outside of sehool variables are 
more important variables for explaining students’ mathematies aehievement, we 
retained variables that aeeounted R-squared >0.02 (see our explanation in the 
Method seetion) and ordered these variables using Relative Pratt Index. Table 
8 lists the results of this variable ordering. Overall, self-eonfidenee (SLCONF) 
was the most important faetor related to students’ aehievement aeross nations 
exeept Singapore where SLCONF was the seeond most important variable. 
Students’ edueational aspirations relative to their parents’ edueation (ASP), 
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Table 7 


R-square Accounted for by School and Outside of school Variables across 
Six Nations or Regions 



KOR 

TWN 

JPN 

U.S. 

SGP 

HK 

Outside of school 

0.44 

0.47 

0.35 

0.37 

0.23 

0.22 

(94%) 

(89%) 

(87%) 

(71%) 

(41%) 

(36%) 

School 

0.03 

0.06 

0.05 

0.15 

0.34 

0.39 

(6%) 

(11%) 

(13%) 

(29%) 

(60%) 

(64%) 

Total i?-squared 

0.47 

0.53 

0.40 

0.52 

0.57 

0.61 


Figure 2 

The Pratt-Index Partition of Variances Accounted for by School and 
Outside of school Variables across Six Nations or Regions 


Pratt Index Partition of School vs. Non-school Variables 



number of books at home (NRBOOK), and the availability of eomputers at 
both home and/or sehool (AVLBLCOMP) were identified as important faetors 
for all nations exeept Hong Kong, whieh was eonsistent with the findings from 
Method- 1 . With regard to sehool-assoeiated faetors, teaehers’ pereeption of no 
or few limitations on instruetion due to students (PERCPLIMT) and emphasis 
on homework (EMHM) were also found to be influential variables for U.S., 
Singapore, and Hong Kong as indieated in Method- 1. 
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Table 8 


Importance Ordering of Variables for Six Nations or Regions 


KOR 



TWN 


JPN 




Pratt 

R-sqr 


Pratt 

R-sqr 

Pratt 

R-sqr 

SLCONF 

0.46 

0.21 

SLCONF 

0.40 

0.21 SLCONF 

0.40 

0.16 

NRBOOK 

0.19 

0.09 

ASP 

0.14 

0.07 ASP 

0.21 

0.09 

ASP 

0.10 

0.05 

NRBOOK 

0.13 

0.07 NRBOOK 

0.09 

0.04 

VALUING 

0.07 

0.03 

LOWSES 

0.04 

0.02 PARENTEDU 

0.06 

0.03 

SCHSIZE 

0.05 

0.02 

TUTORING 

0.04 

0.02 AVLBLCOMP 

0.06 

0.02 

AVLBLCOMP 

0.05 

0.02 

AVLBLCOMP 

0.04 

0.02 TCHTIME 

0.06 

0.02 

TUTORING 

0.04 

0.02 

CONTENTACT (write 
equations) 

0.03 

0.02 






VALUING 

0.03 

0.02 






MCFL 

0.03 

0.02 




0.96 

0.44 


0.88 

0.47 

0.88 

0.36 

U.S. 



SGP 


HK 




Pratt 

R-sqr 


Pratt 

R-sqr 

Pratt 

R-sqr 

SLCONF 

0.24 

0.13 

MATHTOPICS 

0.24 

0.13 SLCONF 

0.18 

0.11 

NRBOOK 

0.13 

0.07 

SLCONF 

0.15 

0.09 CLASIZE 

0.18 

0.11 

TUTORING 

0.11 

0.06 

SCHCLIMATE 

0.10 

0.05 GROUPINSTRUT 

0.15 

0.09 

SCHCLIMATE 

0.06 

0.03 

SCHSIZE 

0.06 

0.04 GOODSCH 

0.07 

0.04 

MATHTOPICS 

0.05 

0.02 

NRBOOK 

0.06 

0.03 MCFL 

0.05 

0.03 

CONTENTACT 

(write equations) 

0.05 

0.02 

MCFL 

0.05 

0.03 EMH 

0.05 

0.03 

MCFL 

0.05 

0.02 

EMH 

0.05 

0.03 SCHSIZE 

0.04 

0.02 

ASP 

0.04 

0.02 

CONTENTACT (write 
equations) 

0.04 

Q Q 2 interact (observe 
others’ teaching) 

0.03 

0.02 

EMH 

0.04 

0.02 

ASP 

0.03 

0.02 SCHCLIMATE 

0.03 

0.02 






PROFDEVELOP 



AVLBLCOMP 

0.03 

0.02 

AVLBLCOMP 

0.03 

0.02 (improve students’ 

0.03 

0.02 






problem solving) 



HIGHSES 

0.03 

0.02 

THW 

0.03 

Q Q 2 CONTENTACT (explain 
their answers) 

0.02 

0.02 




CONTENTACT (decide 





PARENTEDU 

0.03 

0.02 

own procedures for 
solving complex 
problems) 

0.03 

0.02 




0.86 

0.45 


0.87 

0.50 

0.83 

0.51 
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Conclusion and Discussion 

The purpose of this study is to investigate how outside of sehool 
faetors, in the presenee of sehool-assoeiated faetors, are related to students’ 
mathematies aehievement for the U.S. and the five top-performing Asian 
nations or regions (i.e., Korea, Taiwan, Japan, Singapore, and Hong Kong), 
and to examine whether the relative role of sehool-assoeiated and outside 
of sehool faetors is eonsistent aeross the five Asian eountries. Henee, this 
study intends to indireetly test the taeit assumption that higher mathematies 
performanee in these top -performing eountries is an outeome of their superior 
sehool systems. This taeit assumption may be unfounded beeause it is unelear 
how mueh of the Asian eountries’ sueeess in mathematies test performanee 
is related to the publie sehool systems and how mueh to the outside of sehool 
faetors. 

The results obtained from two statistieal proeedures are quite similar, 
identifying the similar faetors that have effeets on students’ mathematies 
aehievement. The eluster effeet of the data has little influenee on the relative 
Pratt indiees for two reasons. First, as a result of a large number of sehool 
level variables ineluded in the multiple regression model, the dependenee of 
varianee may have well been (inadvertently) aeeounted for. In addition, the 
eluster effeet has little influenee on the elements needed to eompute relative 
Pratt index, whieh are standardized/? weights, and simple bivariate 
eorrelation. In general, our findings suggest that students’ mathematies 
aehievement is highly related to outside of sehool faetors sueh as students’ 
self-eonfidenee in learning mathematies, students’ valuing the importanee of 
mathematies, the availability of eomputers in sehool and/or home, and extra 
lesson or tutoring aeross all six nations or regions. Using the Pratt Index, 
the pereentage of explained varianee attributable to outside of sehool faetors 
ranged from 36% for Hong Kong to 94% for Korea. Having said that, sehool- 
assoeiated faetors are found to be related to mathematies aehievement to a 
varying degree aeross the six eountries. Sehool-assoeiated faetors play little 
role in students’ mathematies aehievement for Korea, Taiwan, and Japan. 
However, for Singapore and Hong Kong, the sehool-assoeiated faetors are 
substantially related to students’ performanee and more interestingly, sehool- 
assoeiated faetors have somewhat more important roles than the outside of 
sehool faetors. As for the U.S., sehool-assoeiated faetors are related to students’ 
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performance; however, the outside of school factors clearly play more salient 
roles than the school associated factors. 

It is worth noting that the belief by some policy makers and educators 
that the five TIMSS top-performing countries share a homogeneous 
educational system is an inaccurate stereotype. We have observed similar 
patterns shared by Korea, Taiwan, and Japan where the school-associated 
factors play an insignificant role. Similar patterns are found in Singapore and 
Hong Kong where school-associated factors play a relatively more salient 
role in mathematics achievement. It should be noted that the relative role of 
in and out of school factors is not related to the ranking of these Asian nations 
or regions in TIMSS mathematics tests. For instance, Korea ranks second 
after Singapore and ahead of Hong Kong, but Korea does not share a similar 
pattern with Singapore and Hong Kong. Any researchers or educational policy 
makers who are interested in international education models should note that 
the role of school-associated factors varies across these top-performing Asian 
nations while the outside of school factors play a salient role for all of them. 
Therefore there is no unitary “Asian Model.” 

In the past two decades, U.S. educators and policy-makers have explored 
the Japanese educational system and paid great attention to Singaporean 
education, trying to look for a solution to improve the performance of their 
students in mathematics. The past and current efforts in the U.S. to incorporate 
aspects of the Singaporean school system seem to have some empirical support 
judging by the findings in Tables 5 and 6 that the U. S. achievement attribution 
pattern is closer to those of Singapore. However, our findings reveal that earlier 
efforts in adapting the Japanese educational model was less justified because 
only 13% of explained variation in the Japanese students’ performance is 
related to school-associated factors and only one school associated variable 
in HLM models is related to students’ performance. 

The recent attempts in adapting Singapore’s educational model may 
appear to be more promising. This statement does not imply that incorporating 
Singaporean school model would guarantee an improvement in students’ 
mathematics performance in the U.S. Instead, what we are suggesting is that 
if any of the attempts to incorporate an Asian education model are to be made 
by the U.S. policy makers and reform specialists, then a close look into the 
appropriateness of incorporating a Singaporean or even Hong Kong model 
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would be relatively more promising than other Asian models. It is important 
to keep in mind, however, that our statistieal models were aimed at aeeounting 
for variation within the six nations or regions instead of the differenees among 
the six eountries or regions. Therefore, one should keep in mind that not all of 
the sehools in, for example, Singapore were superior performing sehools - that 
is, there is within eountry variation, whieh we did not investigate in this study. 
Future researeh may find it fruitful to eontrast high and low performing sehools 
in eaeh of these eountries to better understand the edueational proeesses. 

It is, of eourse, very important that one keep in mind that our findings 
should be interpreted as assoeiations or relationships among variables and 
not eausatively. For example, our HLM analyses show that sehool size is 
positively related to Singaporean and Hong Kong students’ aehievement, and 
so is elass size. An intuitive, but ineorreet, eonelusion would be that inereasing 
elass or sehool size would inerease students’ performanee. Sueh a eausative 
statement is not supported by our analyses and data. For example, a simple 
alternative rival hypothesis to the elass or sehool size eonelusion eould be 
that well-performing sehools attraet more students; henee, as a result, sehool 
size and elass size get larger. A poliey deeision based on eausal interpretation 
with eorrelational data is eertainly unjustified. 

Finally, we would like to restate that the elassifieation of sehool- 
assoeiated (i.e., teaeher and prineipal) and outside of sehool faetors eould 
raise some eoneem about our interpretations. We frankly aeknowledge that 
it is possible that different elassifieation may lead to different eonelusions 
and interpretation. Our elassifieation was based on the design of TIMSS data 
eolleetion and also determined by the teehnieal requirement of our data analysis 
(i.e., HLM). Although HLM is eapable of teehnieally dividing variables 
into sehool-assoeiate and outside of sehool variables by the data strueture, 
it eould be praetieally impossible to meaningfully distinguish an outside of 
sehool variable from a sehool-assoeiated variable. For instanee, the sehool 
soeioeeonomie status (SES) is the average or sum of individuals’ SES, and 
henee it is unreasonable to think that the sehool level SES effeet is totally 
distinet from individuals’ SES. Another example is the variable that surveyed 
teaehers’ pereeption of no or few limitations on elassroom instruetion due to 
student faetors (PERCPEIMITATION). Beeause this variable was eolleeted at 
the teaeher/elassroom level, as a result, we eould only model it at the seeond 
level. 
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Our central message to the readers is to remind them that a eountry’s 
edueational system is not an isolated eomponent from the soeiety, but is 
deeply embedded in the eulture, family and personal eharaeteristies, as well 
as the polities and eeonomy of a eountry. Our empirieal findings remind us 
that mathematies aehievement is related to many (if not all) of these variables. 
Students do not study within a vaeuum of a sehool, rather, are living organisms 
that interaet with other elements of their sehool and non-sehool environments. 
As obvious as this seems, as edueational researehers and edueational testing 
speeialists we still sometimes earry about our researeh and edueational poliey 
development taeitly assuming otherwise. 
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Appendix 

Table lA 


Multilevel Estimates for Mode-C (Student-principal Model) 



Parameter 



Coefficient 



School-model 


KOR 

JP 

TW 

U.S. 

SGP 

HK 

Fix effect 
Intercept 

foo 

570.5*** 

529.5*** 

602.3*** 

434.7*** 

495.6*** 

442.1*** 

LOWSES 

foi 

-5.3** 

-4.8 

-16.2*** 

^ Q*** 

-4.8 

-2.2 

HIGHSES 

y 02 

2.8*** 

1.8 

8.2* 

5.5* 

3.8 

5.5 

GROUPCURRIC 

ULUM 

f03 

-2.9 

-2.3 

5.9 

3.2 


-31.1** 

GROUPSTUD 

f04 

3.8 

7.2 

-4.7 

3.8 

4.0 

4.7 

SCHCLIMATE 

7 05 

6.2* 

9.5 

4.1 

8.1 

24 7*** 

15.5 

GOODSCH 

7o6 

0.9 

3.7 

-7.6 

10.48** 

6.0 

22.3* 

SCHSIZE 

7 01 

0.01*** 

0.0 

0.0 

0.0 

0.03*** 

0.09** 

PARENTEDU 

7m 

0.3 

6.0*** 

1.6 

2.3*** 

0.8 

-1.6*** 

ASP 

7 20 

11.8*** 

13.5*** 

-|7 4*** 

5.8*** 

7.0*** 

1.5 

VALUE 

7 20 

9.5*** 

5.6*** 

7.0*** 

-2.52* 

7.5*** 

5.1*** 

SLCONF 

740 

40.1*** 

35.1*** 

43.8*** 

27.4*** 

22.2*** 

25.2*** 

TMHW 

7 50 

-3.4* 

-10.6*** 

4.3** 

10.3*** 

13.4*** 

0.3 

TUTORING 

7eo 

-4.0*** 

3.1** 

-5.6*** 

17.6*** 

3.1* 

7.8*** 

AVABLCOMP 

7 10 

12.8*** 

y Y*** 

^ Q*** 

6.1*** 

10.1*** 

4 4** 

NRBOOK 

7so 

12.3*** 

7 4*** 

^ ^ 2*** 

8.0*** 

6.1*** 

1.1 

Random effect 
Level- 1 variance 


3,179*** 

3,048*** 

3,657*** 

2,140*** 

2,168*** 

1 ,366*** 

Level-2 variance 

^lo 

151*** 

604*** 

1,012*** 

846*** 

659*** 

1,792*** 

Goodness-of-fit 

Deviance 

49,786 

35,196 

51,388 

54,376 

48,371 

32,097 


Note: *** denotes a significanee level at .001, ** at .01, * at .05. 

The table only reported the level-2 variance of random effect, as no 
variable was included into the random coefficient as outcome equations 
except intercept. 
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Table 2A 

Multilevel Estimates for Mode-C (Student-teacher Model) 

Parameter Coefficient 


Teacher-model 


KOR 

JP 

TW 

U.S, 

SGP 

HK 

Fix effect 
Intercept 

Too 

540.8*** 

485.3*** 

530.1*** 

327.5*** 

267.0*** 

305.2*** 

PERCPLIMIT 

rm 

7.3*** 

6.3 

10.3* 

19.8*** 

26.4*** 

15.6* 

EMHW 

y 02 

-0.3 

4.5 

4.1 

27.5*** 

19.1*** 

15.9* 

CLASIZE 

7 02 

6.6** 

4.5 

2.6 

-4.7 

5.1 

38.3*** 

MATHTOPICS 

Yoa 

0.0 

-0.1 

0.3 

1.3*** 

3.4*** 

1.3** 

TCHTIME 

7o5 

0.1 

0.3*** 

0.1 

0.0 

-0.3*** 

0.0 

INTERACTION 1 

7o6 

-1.6 

0.6 

-3.5 

-1.9 

-3.1 

-3.2 

INTERACTION 2 

7 01 

0.5 

0.2 

-1.9 

3.2 

-3.7 

6.0 

INTERACTION 3 

7 08 

2.9 

0.3 

-5.9 

-17.4*** 

-9.9 

-9.6 

INTERACTION 4 

7o9 

1.2 

-5.0 

6.5 

3.9 

0.7 

8.1 

PROFDEVELOP 1 

7oio 

4.7 

-6.9 

6.6 

11.4 

1.8 

-5.3 

PROFDEVELOP 2 

Yon 

-0.3 

7.4 

-11.1 

8.3 

-21.2* 

-12.9 

PROFDEVELOP 3 

You 

3.5 

-1.8 

-11.5 

-12.5 

19.7* 

3.2 

PROFDEVELOP 4 

Yon 

-2.3 

-1.2 

-19.1* 

-5.3 

9.4 

-9.8 

PROFDEVELOP 5 

Yoia 

2.0 

-1.3 

0.5 

3.8 

11.6 

6.9 

PROFDEVELOP 6 

Yois 

-6.6 

-0.2 

8.2 

-3.5 

-13.1 

-6.1 

CONTENTACT 1 

Yo16 

4.2* 

0.0 

-1.8 

0.7 

-2.8 

-10.9 

CONTENTACT 2 

Yon 

-2.8 

1.5 

-1.4 

-1.2 

6.6 

9.8 

CONTENTACT 3 

Yon 

4.4 

2.8 

5.5 

0.7 

-2.8 

-4.3 

CONTENTACT 4 

Yon 

-1.8 

-7.0 

-14.0 

-6.2 

-34.9** 

-7.3 

CONTENTACT 5 

Y 020 

-1.0 

5.3 

21.6*** 

5.2 

19.9*** 

11.4 

CONTENTACT 6 

Yo21 

-2.0 

-6.2 

-1.1 

3.7 

11.0 

-4.6 

CONTENTACT 7 

Y 022 

-2.5 

1.3 

-11.5 

-19.7*** 

5.2 

-0.5 

CONTENTACT 8 

Y 021 

0.6 

4.1 

-5.2 

-5.8 

-6.8 

-3.0 

CONTENTACT 9 

Yo2A 

-0.8 

1.0 

8.6 

9.0* 

10.6* 

-0.1 

PARENTEDU 

Yio 

-0.2 

5.2*** 

2.0 

1.9** 

-0.7 

-1.0 

ASP 

Y 20 

13.0*** 

13.3*** 

18.0*** 

2.5*** 

-0.8 

0.8 

VALUE 

Yio 

1 'i 4*** 

4.5*** 

6.1*** 

-1.4 

8.2*** 

5.1*** 

SLCONF 

Yao 

38.3*** 

35.6*** 

44.1*** 

22.6*** 

17.9*** 

23.4*** 

TMHW 

Yso 

-1.2 

-11.3 

4 8*** 

-1.3 

0.6 

0.9 

TUTORING 

Yoo 

-3.5*** 

3.2** 

-5.7*** 

14.9*** 

3.5*** 

8.5*** 
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AVABLCOMP 

^70 

12.5*** 

y Y*** 

9.3*** 

3.3*** 

3.9*** 

2.9 

NRBOOK 

/so 

12.8*** 

Y Y*** 

11.5*** 

6.0*** 

0.7 

2.4*** 

Random effect 








Level- 1 varianee 

2 

(J 








^ a 

3,124 

3,038 

3,731 

1,504 

853 

1,243 

Level-2 varianee 

< 

226 

354 

887 

1,902 

2,839 

1,658 

Goodness-of-fit 

Devianee 

30,611 

34,435 

48,372 

45,390 

42,971 

37,541 


Note: *** denotes a signifieance level at .001, ** at .01, * at .05. 

The table only reported the level-2 varianee of random effect, as no 
variable was included into the random coefficient as outcome equations 
except intercept. 
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